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ABSTRACT 

The relevance of testing to curriculum is measured by 
the closeness of fit between test items and major program objectives. 
In this paper two procedures for analyzing the curriculum/test 
overlap are compared. The first method obtains a detailed analysis of 
a priority area of the curriculum. The process brings the curriculum 
into alignment with the test. It is a structured, sequential, 
rational, and objective method to be conducted by the classroom 
teacher. However, without structured available time and guidance from 
district administration, the analysis will probably not be 
accomplishes. The process requires a long period of time to achieve a 
comprehensive appraisal o£ the match between the district assessment 
tool and curriculum. The second method involves an independent 
consultant who assesses the degree to which the district assessment 
tool measures objectives across content are** and grades. The results 
are presented by subject area and gra^e level along with a statement 
of recommendations. This method results in a comprehensive assessment 
of the overlap between curriculum objectives and test items. The 
process is subjective and prone to inconsistencies. Achievement of 
changes in classroom practice by teachers is primarily dependent upon 
the district leadership's ability to see that innovations are 
implemented. (DWH) 
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Curriculum/test overlap is an educational issue of considerable 
importance. Assessing curriculum/test overlap is one activity which 
characterizes an effective school. It is necessary to be able to speak 
to the effectiveness of an instructional proaran; based on test data. 

The ma; or reason we measure achievement is to determine whether or 
not the instructional program is TT>eeting writtor or unwritten 
objectives. To do this requires answers to two very important 
questions: (1) Are we testing what we teach? (2) Do we teach what we 
say we teach? This paper's focus is on the first question — that of 
relevance. The relevance of the test to the curriculum is measured by 
the closeness of fit between tha items making up the test and the major 
objectives of the program. It should be the major criterion for test 
selection and use. 

Two projects to assist two Alaskan school districts assess their 
curriculum/test overlap were completed by the author. The two projects 
resulted in v -ry different procedures for analyzing the curriculum/test 
overlap and, eventually, different outcomes for the distri* ■;. This paper 
presents a compariscn of the two approaches. The advantages and 
disadvantages of each will be discussed. The tv^ client districts will 
be referred to as School District A and School District B. 

Checking for Curriculum/Test Overlap in School District A 

In School District A (SDA) , there was a concerted effort to 
strengthen and improve the schooling of it? students. District goals and 
objectives had been developed and, with the assistance of a USED/ESEA 
Title Basic Skills grant, SDA had directed its attention toward 
studeri ...•:> ' needs in the basic skills. 

SDA requested assistance in helping teachers link test results from 
their district assessment to curiiculum improvements^ and ultimately, 
increases in student achievement. The SDA used the Iowa Test of Basic 
Skills (ITBS), published by Riverside Publishing, 1980. The 
curriculum/test overlap request was initiated by the Director of 
Elemencary Education, who also managed the Titlt II grant. The 
cur r iculurii/test overlap study was only one of many activities that had 
been initiated to improve student ach:evemer.t in the basic skills, 
including training of administrators and teachers in effectiveness 
schooling research and techniques to increase student time-on- task . 

SDA was led to request a process for analyzing curriculum/test 
overlap because of the very low student achievement scores on the ITBS. 
The Director of Elementary Education felt that there must be a way 
teachers could use the test information to assist in improving student 
achievement but did not know how to approach the problem. 



The author made an onsite visit to further clarify the problem and 
determine the best approach to meeting the need. Teachers were 
interviewed for their current use of test data. It was discovered during 
this preliminary step that the teachers did not use the test data for 
decision-making about what and how they taught and expressed frustration 
over not really understanding the data, or how to begin to use it. 

The clarification process led the author and director to conclude 
that it was a three-step problem: (1) teachers needed to know how to 
administer tests to students and how to assist students in taking tests 
(without, of course, unethical procedures); (2) the teachers needed to 
understand the meaning of the data they received on their students; and 
(3) they needed to know what to do about the data they received. 

Cla r if icat ion of the teache rs ' needs relative to the test ing process 
and data interpretation and use made approaching the ^solution 
self-evident. The author and a colleague developed a guide for using 
test information to improve instruction. The main sections of the guide 
covered test administration skills for the teacher, test taking skills 
for the student, reading test reports, using test information for 
curriculum development and implementing direct instruction. It is the 
section on using test information for curriculum develoment which 
addresses curriculum/test overlap and presents a procedure by which to 
assess the overlap. 

The teachers were trained in the application of the process. The 
training developed the capacity for each teacher to assess his/her own 
degree of curriculum/test overlap. 

The curriculum/test overlap assessment presented to SDA in the guide 
comprises four activities. In the first activity, a grade level , prof ile 
of skill weaknesses is created from the ITBS student list report (from 
the scoring service). The grade level profile leads to the 
identification of the weakest subtest skill area on the ITBS for a grade 
level of students. The weak skill areas are ranked, which allows 
teachers to quickly identify areas where students need assistance. 

In the second activity, teachers identify a priority skill area on 
which to focus. A weighted decision-making guide was developed to assist 
teachers in making an objective assessment of the priority skill area. 

Once the priority skill area is identified, the third activity leads 
one through an analysis of that skill area. The skill area is looked at 
in detail to determine exactly what is being tested by the ITBS. This is 
accomplished through a careful review of the Building Criterion^ 
Referenced Skills Analysis. 1^ te'sults in a rank ordering of the 
subskill areas by weakest to strongest. 

The fourth activity results in the overlay of curriculum onto the 
test. This step is less obj ect i ve" ; no hard data are available to 
determine when a skill is being taught. The teacher must review his/her 
curriculum guides and indicate when and where and for how long a skill is 
taug ht . 



Based on this very prescriptive analysis of the curriculurr, and test, 
:n a priority area only, the guide further leads one through a 
development process to remediate any '"holes" in the overlay. It is a 
process by which one brings the curriculum into aZicnr.ent with the test, 
rather than leading to a single judgement about whether or not there is 
alignment. The process is structured, sequential and objective. It was 
to r-e dene by the classioom t::acher. 



Ac vantages 

This approach to assessing curriculum/test overlap has the advantaae 
of being rational and objective. Because it is a structured and 
sequenMal process, there is an inherent degree of reliability in the 
judgements that will be made about the curriculum/test overlap. It is 
done by teachers who are familiar with the curriculum objectives and what 
is really being taught, thus increasing the reliability of the 
assessment. It is anticipated that going through such a step-by-step 
process could result in immediate changes in teaching practices that 
might result in improvements in achievement. By working with priority 
skill areas, it allows for a manageable approach to making chc.nges in the 
curriculum. Once the priority skill area has be'en adequately addressed, 
the next priority skill area will be reviewed. 



D isad vantaaes 



There are two primary disadvar. -ages of such an approach to assc^ssing 
curriculum/test overlap. Without structured available time and guidance 
from the school district administration, it is likely the analysis will 
not be accomplished. The demands on teachers' tiipe and attention may not 
lead to this becoming a priority activity in which to engage. 

The second main d i 5; ad vantage results from th^ adva-.tage of a "bite 
size" approach.- Except • ver time, the process does not result in a 
comprehensive appraisal of the match between the d ist : ict-wide assessment 
tool and the district curriculum. 

In addition, there was a change of personnel in SDA soon after the 
training of teachers in the process. Attention has shifted away from 
pursuing the completion of the curriculum/test overlap/development 
process. This hazard exists in any work situ.tion. For the process to 
have any chance of impact, it will have to be fully implemented. 

Checking for Curriculum/Test Overlap in School District B 

School District B (SDB) requested a comprehensive analysis of the 
rr.atch between their district curriculum (all subject areas, all grades) 
and their district assessment tool (the Iowa Test of Basic Skills — ITBS 
and the Tests of Academic Proficiency — TAP). The teachers had expressed . 
concern over the significantly low scores their students were receiving 
on the ITBS and TAP. They were concerned about whether or not the low 
scores may have reflected a poor match between what they taught and what 
was being tested. 



The district director of instruction contracted to have *"he analysis 
Gone. The author of this paper conducted the study. The purposes of the 
assessiTient were (1) to assess the degree to which the ITB5/TAP test items 
n>^asure SD3 objectives acros.-; content areas and grades, and (2) to 
provide recommendations about those findings (i.e., implications for 
interpreting IT3S results, observations on or revisions of SDB objectives 
and possible next steps to be-:t utilise ITBS/TAP or other test 
information) . 

The reviewer read each set of SDB objectives. The corresponding 
ITBS/TAP test items were then reviewed. Items which the reviewer judged 
to assess an objective were noted under the objective. Items left 
unassigned to an objective were noted in the margin. Two ratios wore 
computed where possible: (1) number of ITBS/TAP items unassigned to the 
total number of ITBS/TAP items available, and (2) number of SDB 
objectives by grade level and by content area measured by at least one 
item to the total number of SDB objectives within that content area. 
These ratios were expressed as percents. Tables were created containing 
the ratios of actual numbers of items. The first ratio indicated how 
much of the ITBS/TAP was not •'used." This provided an indication of the 
efficiency of the ITBS as measured by the degree to which its items 
measured the SDB objectives. Conversely, a second ratio was used to 
suggest the portion of the SDB curriculum addressed by the ITBS, This 
was to allow consideration of a review or rewrite of the SDB objectives, 
or that some objectives may need to be assessed in other ways (i.e., 
criterion-referenced te^ts, performance tests^ observations, surveys, 
etc. ) , 

The results of the analysis were presented by subject area and by 
grade level in a two-voluTne report to SDB. A brief narrative described 
the data displayed in an accompanying table of results. Any ' impl icat ions 
were discussed, and recommendations were made within the content area of 
analysis. A final statement of recommendations was made at the close of 
the report. The worksheets were included in an attached appendix. 

Advantages 

This particular approach to the curriculum/test overlap presents two 
distinct advantages, especially in comparison to that used with SDA. 
First, the curriculum/test overlap analysis was completed in its 
entirety. Second, it resulted in a comprehensive assessment of the match 
or overlap between their curriculum objectives and test items. 

By contracting with an outside agency which could assign an 
individual to perform the analysis, SDB was assured of having the 
analysis completed. As seen with the in-district approach presented to 
SDA, it is all too likely, with the variety of issues prt:'i^^ing on =^ 
district*^ resources (time, money and people), that the analysis ■ ^1 
never be done. If a district cannot structure time to have teachers 
perform the analysis for use of the information in their classrooms^ then 
it seems to present a higher probability of success for either an 
ir-district individual to be assigned to do the task (which requires a 
sotting aside of other duties) or an outside consultant to be hired for 
the job. 
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Because the jot was contracted, the district was able to achieve a 
conprehensi ve analysis of their curriculum objective/test iter?, overlao. 
In contrast, SDA would have achieved an analysis of only part of eir 
cur r iculurr,. To aet a comprehensive analysis of the cu r r iculurr^/tebc 
overlap in SDA would have required several "rounds" of effort. The net 
result of the approach taken by SDB was a coirprehensi ve measure of the 
overlap at all grades, for all content areas. 

D j sad va ntaoes 

There are a few limitations to the study which must be considered 
when interpreting the results. The limitations, however, do not 
invalidate the results. 

The first is the inherent s^hjectivity of the process of assigning 
items. The process of assignment, was a decision on the part of one 
person, removed from the school district, determining if an item did or 
did not assess a given objective. The distinction was, more often than 
not, unclear. 

The subjectivity leads to the second limitation — consistency. The 
task of assignment requires consistency in reasoning and judgement. Any 
individual or group will encounter these problems. Inconsistencies or 
shifting in standards of assignment were noted by the reviewer. 

A third disadvantage is that to achieve any change in classroom 
practice on the part of teachers will require greater effort by the 
district. In the approach taken by SDA, teachers have first-hand 
experience with seeing when what they teach, or do not teach, is or is 
not tested. In SDB, teachers will have to be informed of the matches and 
discrepancies.' In both school districts, some amount of administrative 
leadership will have to be exhibited to achieve a change in classroom 
practices. 

InSDB, persistent administrative leadership has led to significant 
changes being set into motion as a result of the curriculum/test 
overlap. The report on the overlap was used to convince others of the 
need to revise their curriculum. In their words, ''The analysis helped us 
decide to treat our curriculum objectives as a first draft and that they 
should be revised." SDB will probably keep the ITBS/TAP test system 
because ot a strong longitudinal data base unavailable anywhere else in 
the state. 

The revised curriculum will be put on microcomputer to create i data 
base. An item analysis will be on line to allow teachers to see which . 
objectives are measured by which items and how their students are 
performing on those objectives. The system will also indicate what 
objectives are taught at which grade level, and the instructional 
r-.aterials and software available to teach those objectives. The total 
system is expected to be completed within a year. 



S uT.mary and Cone 1 us ions 



One character istiir of an effective school is its attention to 
curriculurr. alicatient and achieverrient data. This paper has discussed two 
nethcns by which two Alaskan school districts focused their time and 
attention onto curriculuin alignment and test data. The two methods 
resulted in two very different outcomes for the respective d ist r icts--one 
achieved a comprehensive analysis of its curriculum/test overlap by an 
outside consultant and one obtained a very detailed analysis, by 
classroom teachers, of a priority area of the curriculum • 

Advantages and disadvantages accompany both methods. The difference 
in the usefulness of the two methods, however, rests with the leadership 
to see them put to use. 
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